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EXECUTIVE  SUMMARY 


This  report  presents  an  evaluation  of  methods  used  for  distribution,  storage,  and 
display  of  satellite  pictorial  products  in  the  Model  2  Flight  Service  Automation 
System.  Project  objectives  include  the  test  and  evaluation  of  currently  available 
data,  presentation  methods,  operational  procedures,  and  system  configurations.  In 
accomplishing  these  objectives,  the  question  of  cathode-ray  tube  (CRT)  versus 
hardcopy  display  of  weather  products  is  addressed.  Additionally,  the  CRT  display, 
with  enhancements  to  demonstrate  its  suitability  as  a  tool  in  the  Flight  Service 
Station  (FSS)  briefing  function,  was  investigated. 

Results  of  this  phase  1  study  have  shown  the  preference  for  and  complete  adequacy 
of  the  CRT  display  of  satellite  images  over  that  of  contemporary  hardcopy.  In 
addition,  digitally  enhanced  infrared  images  were  preferred,  together  with  the 
capability  to  project  these  images  at  medium  speed,  slow  speed,  and  individually 
(step  feature);  the  speed  depending  on  the  specialist's  current  requirements. 
Finally,  and  most  importantly,  the  animation  feature  was  the  most  praised  and 
desired  of  all  features  demonstrated  which  utilized  the  Geostationary  Operational 
Environmental  Satellite  (GOES)  system  products. 

It  was  concluded  that  the  proposed  satellite  image  display  system  was  a  more 
effective  tool  when  augmented  by  other  information  and  products  for  overall 
briefing  effectiveness.  Hence,  it  is  recommended  that  a  phase  11  field  test  be 
conducted  for  further  system  developmental  input. 


INTRODUCTION 


PURPOSE. 

The  purpose  of  the  test  was  to  evaluate  methods  for  the  distribution  and  display  of 
satellite  pictorial  products,  as  set  forth  in  the  specifications  for  the  Model  2 
Flight  Service  Information  System  Automation  Program. 

BACKGROUND . 

As  part  of  the  Flight  Service  Automation  System  (FSAS),  a  preliminary  evaluation  of 
a  prototype  distribution  and  display  system  for  weather  satellite  images  from  the 
Geostationary  Operational  Environmental  Satellite  (GOES)  system  was  conducted  at 
the  Federal  Aviation  Administration  (FAA)  Technical  Center  by  personnel  of  the 
Flight  Service  Station  Branch  (ACT-250)  during  the  period  September  4,  1980,  to 
October  17,  1980,  utilising  24  field  Flight  Service  Station  (FSS)  personnel  from 
all  regions  within  the  FAA. 

SYSTEM  DESCRIPTION. 

The  engineering  model  being  used  for  phase  1,  in-house  evaluation,  automatically 
and  continuously  receives,  stores,  and  processes  satellite  images  which  are 
transmitted  over  telephone  lines  to  the  subscriber. 

The  recorder  stores  images  in  standard  television  format.  A  total  of  96  images  are 
recorded,  two  per  hour,  with  only  the  last  48  hours  being  retained  in  a  dual- loop 
pictorial  record.  As  each  new  image  is  processed  and  recorded,  the  oldest  image  in 
the  appropriate  loop  is  deleted.  Hence,  the  recorder  stores  the  images  on  two 
"movie  loop"  tracks. 

The  user/specialist  can  view  the  stored  images  in  many  different  ways.  That  is, 
the  entire  48  hours  as  a  "movie”  can  be  reviewed,  played  forward  or  backward  at 
one  of  the  three  sequencing  rates,  or  played  through  forward  or  backward, 
image-by- image,  allowing  "stop  action"  on  any  image. 

Other  special  features  allow  the  specialist  to  display  a  partial  loop  containing 
the  latest  6  hours  of  images.  Also,  the  most  current  image  can  be  called  up  using 
the  STILL  (ST)  display  command.  Sectorization,  although  not  fully  developed,  will 
offer  the  operator  a  3-to-l  blowup  of  preselected  images. 

FUNCTIONS  AND  FEATURES.  The  program  written  for  the  in-house  phase  I  evaluation  of 
the  GOES  Distribution  and  Display  System  (GDDS)  enables  the  user  to  move  freely 
within  the  systems  software  framework  regardless  of  where  he  enters  or  where  he  is 
in  the  program.  Only  the  first  two  letters  of  each  command  word  are  required  to 
access  a  feature  of  the  system.  Also,  the  whole  word  or  any  part  (in  excess  of  two 
characters)  may  be  entered  —  this  will  get  the  operator  into  the  program.  All 
commands  are  executed  by  the  system  when  the  "Return"  key  is  depressed;  that  is, 
the  command  sequence  is  not  completed  until  the  "Return"  function  is  keyed.  The 
command  functions  are  as  follows: 


1.  Types  of  imagery  displayed. 


a.  INFRARED  (IR)  -  displays  digitally  enhanced  and  standard  infrared  images 
which  are  received  on  the  hour  from  "Loop  1." 

b.  VISUAL  (VI)  -  displays  visual  satellite  images  collected  on  the  half  hour; 
however,  all  night  images  received  on  the  half  hour  will  be  IR  and  are  placed  on 
the  VI  loop  (LOOP  2). 


c.  BOTH  (BO)  -  causes  consecutively  displayed  images  to  come  alternately  from 
IR  (LOOP  1),  then  VI  (LOOP  2),  etc.,  causing  the  recorder  to  operate  as  a  single 
loop  device.  In  order  to  back  the  system  out  of  the  BO  mode,  the  operator  must  key 
in  an  IR  or  VI  command  function. 

2.  Quantity  of  images  available  in  sequence. 

a.  PARTIAL  (PA)  -  displays  the  most  current  images  of  the  selected  type 
received  in  the  last  4  or  8  hours  (16  images  in  the  BO  mode;  8  images  in  Loop  1  or 
Loop  2  mode). 

b.  FULL  (FU)  -  causes  all  images  of  the  selected  type  to  be  presented 
(96  images  in  the  BO  mode;  48  images  in  Loop  1  or  Loop  2). 

3.  Viewing  mode. 

a.  STILL  (ST)  -  causes  the  display  to  stop  and  display  the  most  current  image 
in  the  loop  of  the  type  of  image  previously  selected  when  this  mode  is  entered. 

b.  ANIMATED  (AN)  -  causes  the  presently  selected  rack  of  images  to  be 
displayed  in  an  animated  and  forward  sequencing  mode  at  a  specified  rate  (FAST, 
MEDIUM,  SLOW). 

4.  Other. 

a.  FORWARD  (FO)  -  causes  the  currently  selected  rack  to  be  displayed  or 
sequenced  in  the  forward  mode. 

b.  REVERSE  (RE)  -  causes  the  currently  selected  rack  to  be  displayed  in  a 
backwards  (reversed  order)  time  or  "stop  action"  playback  at  any  selected  rate. 

c.  HALT  (HA)  -  causes  the  animated  display  to  stop  and  enter  the  single  step 
mode.  It  is  suggested  that  this  function  be  used  when  scanning  for  a  specific 
iauige;  enter  HA,  when  the  desired  images  appears  on  the  display  key  in  the  (CR) 
function  to  execute  the  command.  If  the  system  overruns  the  desired  image,  it  is 
easily  retrieved  by  using  the  FO/RE  commands. 

EQUIPMENT  INSTALLATION. 

Animation  and  display  of  GOES  imagery  was  obtained  by  the  use  of  an  Information 
Processing  Systems  (IPS)  video  disc  unit.  This  device  was  controlled  by  an 
Interdata  7/32  minicomputer. 


2 


aaaaa 


SalE  •  **  ■ 


The  IPS  unit  received  its  satellite  imagery  input  from  a  half-duplex  C-2 
conditioned  data  circuit.  No  control  of  images  received  at  the  test  site  was 
possible  owing  to  the  nature  of  this  line.  Incoming  images  received  between  5 
minutes  before  and  25  minutes  past  the  hour  were  stored  on  "Loop  2"  of  the  IPS  unit 
and  were,  in  general,  digitally  enhanced,  stretched  gray  scale,  infrared  products. 
Those  images  received  between  25  minutes  and  55  minutes  past  the  hour  were  stored 
on  "Loop  1"  of  the  IPS  unit  and  were  generally  visual  images  for  daylight  hours  and 
unmodified  infrared  during  the  nighttime  period.  Forty-eight  images  could  be 
stored  on  each  track. 

The  video  disc  also  served  to  convert  the  GOES  information  from  the  audio  to  the 
video  domain  via  a  digital  scan  rate  conversion  technique.  Excessive  information 
for  video  display  was  also  rejected  during  the  conversion.  The  resulting  product 
of  the  conversion  process  was  stored  on  the  video  disc  in  analog  format. 

The  display  device  employed  for  the  test  was  a  14-inch  diagonal  measure  P43 
phosphor  CRT  display.  The  unit  was  not  selected  as  an  optimum  display  device.  The 
choice  was  determined  by  the  availability  of  monitors  during  the  test  period.  This 
display  also  functioned  as  a  feedback  device  to  the  subject.  Switching  between  the 
two  modes  of  operations  was  achieved  via  a  manual  switch  located  at  the  test 
position. 

Operator  input  was  by  means  of  a  standard  ACSI1  keyboard  connected  via  an  E1A 
standard  RS-232C  channel  to  host  computer.  A  control  program  decoded  the  oper¬ 
ator's  simple  English  commands  and  sent  the  corresponding  command  code  sequence  to 
the  IPS  unit.  The  control  link  between  the  video  disc  and  the  host  computer  was 
half  duplex,  EIA  standard  RS-232C. 


SYSTEM  EVALUATION 


MAJOR  LIMITATION  OF  THE  EVALUATION. 

All  test  subjects  were  experienced  journeymen  flight  service  station  specialists 
with  an  approximate  equal  mix  of  subjects  from  automated  and  nonautomated  flight 
service  stations.  However,  to  enable  the  reader  to  judge  the  validity  of  the 
conclusions  reported  from  the  data  and  the  general  value  of  the  evaluation,  it  is 
important  to  point  out  the  following  major  limitation.  Of  the  24  specialists 
selected,  only  three  specialists  had  formal  training  in  satellite  imagery  interpre¬ 
tation,  and  just  two  were  utilizing  this  training  and  the  GOES  products  in  their 
home  facilities.  However,  it  was  felt  that  their  evaluation  of  the  proposed  system 
and  concept  would  be  of  distinct  value  in  light  of  their  substantial  FSS  experience 
and  weather  knowledge. 

EVALUATION  PROCEDURES. 

The  GOES  system  evaluation  period  took  approximately  2  hours,  and  included  a 
specialist  questionnaire  (appendix  A)  and  interviews  by  a  project  evaluation  team 
member  (appendix  B).  The  intent  of  the  interview  was  to  be  an  open  forum  for  the 
evaluating  specialist  to  discuss  with  a  Technical  Center  engineering  research 
psychologist  and  a  senior  project  FSS  specialist,  his  impression  of  the  GOES  images 
as  a  tool  for  use  in  FSS  pilot  briefings.  Comments  were  also  solicited  on 
improving  the  proposed  system. 


Data  collected  included  both  an  engineering  evaluation  of  equipment  and  a  uaer 
evaluation  of  display  techniques  and  products.  In  addition,  since  the  requireaent 
to  interface  with  existing  and  planned  operational  displays  was  imposed  on  the 
system  design,  data  collection  and  analysis  addressed  how  well  the  operational  and 
human  factors  aspects  were  handled. 

USER  EVALUATION.  The  user  evaluation  was  conducted  in  as  realistic  a  setting  as 
possible,  utilizing  FSS  journeyman  specialists  from  the  field  to  evaluate  the 
system.  Specifically,  24  specialists  of  various  background,  training  levels,  and 
facilities  (Levels  1,  II,  and  111)  were  shown  contemporary  satellite  products  and 
were  then  asked  to  evaluate  those  products  observed  with  the  display  system 
assembled  for  this  study. 

The  specialist  filled  out  a  questionnaire  for  the  purpose  of  evaluating  both 
display  types  and  methods.  This  was  followed,  where  appropriate,  based  on  GOES 
training  and  field  facility;  i.e.,  automated  or  manual,  by  a  personal  interview  for 
the  purpose  of  obtaining  a  general  assessment  of  the  GOES  products  and  displays. 
Project  personnel  were  also  available  to  assist  the  specialists  during  the  evalu¬ 
ation  and  to  make  observations.  In  this  capacity.  Technical  Center  project 
personnel  documented  their  own  comments  and  those  comments  made  by  the  specialists 
during  the  test  evaluation  period.  They  also  noted  suggestions  made  by  the 
specialists.  These  comments,  suggestions,  and  observations  were  considered  when 
evaluating  test  data  and  formulating  test  results. 

ENGINEERING  EVALUATION.  The  engineering  evaluation  and  questionnaire  data  of  an 
objective  multiple-choice  nature  were  collected,  summed,  and  subjected  to  appro¬ 
priate  statistical  analysis.  And  finally,  the  collected  questionnaire  and 
interview  data  of  a  subjective  nature  were  ranked  and  a  rank-order  correlation  was 
derived  for  specialist  preferences. 

Table  1  is  a  tabulation  of  the  responses  to  the  GOES  System  Specialist  Opinion 
Survey  form.  The  form  used  is  shown  in  figure  1.  This  survey  is  a  qualitative 
comparison  between  the  contemporary  means  of  receiving  and  displaying  these 
products  and  the  proposed  video  displays.  Following  a  demonstration  and  hands-on 
experience  with  the  prototype  system,  the  specialists  would  express  their  opinion 
on  aspect  items  of  the  survey  form  as  to  whether  a  particular  aspect  was  an 
improvement,  remained  the  same,  or  if  the  present  system  was  better. 

STATISTICAL  ANALYSIS. 

To  accomplish  the  statistical  analysis,  the  following  integer  numerical  values 
were  assigned  to  the  five  choices  on  the  opinion  survey:  vast  improvement 
(5),  improvement  (4),  no  change  (3),  present  system  is  better  (2),  present 
system  is  much  better  (1).  The  number  of  responses  within  each  choice  category  was 
weighted  by  the  value  of  the  category  and  combined  using  standard  statistical 
methods.  A  mean  and  standard-error-of-the-mean  were  computed  for  each  of  the 
aspects  within  each  of  the  data  sets.  Student's  "t"  tests  were  performed  to 
determine  the  statistical  significance  of  the  deviation  of  the  mean  response  from 
the  center  of  the  scale  (3  s  no  change).  For  those  aspects  whose  means  were 
significantly  below  3.0,  the  consensus  of  specialists  was  that  the  present  system 
was  better.  If  the  mean  fell  significantly  above  3.0,  an  improvement  in  that 
aspect  was  noted.  Unless  otherwise  noted,  a  confidence  level  of  alpha,  equal  to  or 
less  than  0.05,  was  used  to  determine  significance  of  the  t  score.  Fince  there  was 
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FIGURE  1.  GOES  SYSTEM  SPECIALIST  OPINION  SURVEY  FORM 


TABLE  1.  GOES  SYSTEM  —  SPECIALIST  OPINION  SURVEY 


♦Three  specialists  did  not  respond  to  questions 
1  through  13  and  13. 

Four  specialists  did  not  respond  to  question  14. 


no  a  priori  reason  to  expect  deviations  from  the  mean  in  only  a  single  direction, 
two-tailed  tests  were  used  for  all  items.  All  questions  were  phrased  so  that 
careful  reading  would  result  in  fair  responses,  thus  precluding  stereotype  answers. 
Also,  response  categories  were  varied  so  as  to  minimize  common  sources  of  error, 
such  that  no  response  was  unduly  influenced  by  a  single  or  overall  favorable  aspect 
when  rating  dimensions.  That  is,  the  form  was  designed  so  the  respondents  would 
read  and  respond  to  each  item  independent  of  the  previous  item  or  any  overall 
attitude  toward  an  automated  system. 

The  strong  preference  for  such  a  display  system  is  profoundly  evident  from  the 
statistical  results  presented  in  tables  2  and  3.  Although  the  questions  were 
phrased  to  include  a  balance  of  favorable  or  unfavorable  responses,  the  interpreted 
value  of  the  changes  was  positive  over  93  percent  of  the  survey.  The  aspect  item 
that  yielded  the  highest  t  score  was  item  1  (speed  of  data  access). 

SUMMARY  OF  RESPONSES. 

RESPONSES  FROM  GOES  SYSTEM  EVALUATION  QUESTIONNAIRES. 

1.  Do  you  prefer  machine  display  (as  in  the  IPS)  or  hardcopy  display  of  satellite 
weather  information  for  use  in  pilot  briefings? 

Machine  display,  utilizing  CRT's,  was  the  overwhelming  choice  as  a  display 
medium.  Hardcopy  had  a  singular  benefit  mentioned  by  several  specialists;  namely, 
increased  clarity  especially  on  the  visual  images.  But,  even  with  this  benefit, 
those  specialists  prefer  machine  display  when  looking  at  the  total  system 
utilization.  Pertinent  comments  were  as  follows: 

"The  machine  display  is  much  better  suited  for  pilot  briefings  because  of  the 
looping  capabilities,  ease  in  accessibility,  and  simple  operational  codings." 

"I  feel  the  IPS  (machine  display  under  evaluation)  is  better  suited  for 
briefings  as  it  is  much  easier  to  show  a  trend  and  has  an  easier  access  to  weather 
data." 


"Machine  display  does  not  have  the  clarity  of  hardcopy,  particularly  with 
visual  (VI)  imagery.  However,  advantages  of  machine  display;  i.e.,  "movie  loop," 
outweigh  this  factor." 

"1  prefer  a  machine  system.  In  normal  situation,  the  hardcopies  float  around 
various  positions  making  access  difficult,  if  not  impossible." 

2.  Can  you  adequately  use  the  machine  display  medium  in  the  performance  of  your 
duties  at  prefl ight . . .at  in-f light .. .at  EFAS? 

Response  to  this  question  was  mixed:  73  percent  said  "Yes;"  9  percent  said 
"No;"  and  18  percent  gave  a  conditional,  but  qualified  "Yes."  The  "no"  responses 
were  due  to  lack  of  training/familiarization  associated  with  interpreting  satellite 
imagery . 

Responses  were  highly  favorable  at  the  preflight  position  (100  percent), 
especially  in  face-to-face  briefings.  Responses  were  somewhat  negative  at 
in-flight  and  EFAS.  As  one  GOES  system  trained  specialist  said,  "Real-time  demands 


TABLE  3.  T  SCORE  ANALYSIS,  PART  II 


Respondents  fevor  Video  Display  System 
Respondents  Fevor  Current  System 

The  difference  is  not  statistically  reliable;  therefore,  we  cannot  offer  any  compelling  evidence 
at  this  time  that  there  is  any  profound  difference  in  item  14  relative  to  either  systrsi. 


were  not  met."  The  greatest  value  seemed  to  be  in  the  specialist  self-briefing  for 
an  overall  view  of  the  weather  trend.  In  addition,  it  was  felt  that  GOES  product 
could  be  effectively  used  at  all  positions  and  would  augment  weather  graphical 
products;  e.g.,  surface  analysis  charts. 

3.  What  is  your  reaction  to  the  system's  capability  to  update  satellite  images? 

Responses  ranged  from  "very  good"  to  "excellent."  Update  prior  to  machine 
display  was  left  to  comparison  only.  In  utilising  the  CRT,  the  movement  leaves 
little  doubt. 

4.  What  is  your  reaction  to  the  sequencing  capability  of  the  system?  Do  the 
animated  displays  assist  you  in  weather  trending? 

The  animation  feature  was  the  most  highly  praised  feature  identified  in  this 
GOES  evaluation.  The  animation  and  the  capability  to  control  one's  own  pace 
utilizing  the  step  mode  were  the  major  advantages  identified  in  this  experimental 
GOES  system.  The  animated  displays  were  noted  as  being  a  definite  advantage  (1)  in 
seeing  a  weather  trend,  (2)  for  developing  an  insight  into  weather  causes,  and 
(3)  being  able  to  observe  real-time  weather  moving  over  an  area  (e.g.,  thunderstorm 
development) . 

5.  What  would  be  your  requirement  for  optimum  system  trending? 

Comments  to  this  question  were  broad  and  varied.  Since  21  of  the  24 
specialists  had  not  had  GOES  training,  their  comprehension  of  the  question  was 
limited.  Hence,  only  the  responses  from  those  with  the  formal  training  in 
satellite  interpretation  were  considered.  Their  suggestions  were  for  color 
enhancements,  sectorizing,  and  sequencing  at  the  local  level  and  the  availability 
of  hardcopy,  as  well  as  machine  display. 

6.  How  does  the  quality  of  the  pictures  displayed  in  the  CRT  and  TV  monitor 
compare  with  the  quality  of  the  pictures  from  the  Laserfax? 

It  was  stated  that  when  looking  at  a  single  image  (picture),  the  quality  was 
better  than  on  the  Laserfax.  It  was  more  difficult  to  determine  fine  shade  differ¬ 
ences  on  the  CRT.  It  was  suggested  that  a  change  in  the  CRT  phosphor  from  green  to 
grey  (black/white)  would  improve  the  present  display.  The  CRT  picture  seemed  to 
quiver  slightly,  causing  some  eyestrain  when  viewed  closely.  It  should  be  noted 
that  these  comments  relate  to  individual  or  single  picture  quality,  not  the  overall 
evaluation  of  the  GOES  system. 

7.  What  is  your  preference  as  a  display  medium  —  the  CRT  or  the  TV  monitor  — 
and  why? 

The  CRT  was  preferred  over  the  TV  monitor  due  to  picture  quality  and 
resolution. 

8.  Does  color  enhance  or  detract  from  the  usability  of  the  weather  product  when 
compared  to  standard  black  and  white  displays? 
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No  clear  preference  was  made  for  color  over  black  and  white  displays.  Color 
provided  certain  enhancement,  but  black  and  white  was  preferred  for  clarity  and 
detail.  (Note:  Color  displays  were  not  available  for  evaluation  at  this  time. 
The  question  was  posed  hypothetically. 

9.  What  other  weather  pictorial  overlay  would  best  supplement  the  information  on 
the  satellite  images? 

Specialists  want  to  have  an  overlay  capability  on  the  GOES  maps.  They  would 
like  to  select  the  type  of  overlay  to  suit  the  briefing  requirement.  Overlays  most 
often  mentioned  were  surface,  analysis,  radar  sumary,  200-300-500  MB  chart,  as 
well  as  weather  producing/causing  systems  (highs/lows)  and  fronts. 

10.  What  is  your  reaction  to  the  zooming  capability  of  the  syBtem? 

What  portion  of  the  product  should  be  displayed? 

A  zoom  capability  was  favored  by  all  evaluating  specialists.  It  would  be 
especially  beneficial  for  special  terrain,  such  as  mountainous  areas.  It  would 
also  be  beneficial  for  special  weather  situations,  such  as  thunderstorm  buildups. 
The  preference  was  for  the  ability  to  zoom  in  on  any  sector  of  the  satellite 
picture  that  was  available. 

11.  The  specialists  were  asked  if  they  had  a  preference  for  sequencing  rate, 
degree;  i.e.,  time  lapse  and  preferred  image  type  (digitally  enhanced  infrared 
image  or  a  standard  visual  product). 

The  consensus  of  the  group  indicated  a  medium-to-slow  sequencing  rate  on  the 
order  of  four  frames  per  second.  The  choice  of  product  was  clearly  for  an  infrared 
image.  Finally,  a  clear  choice  of  3-,  6-,  12-,  or  24-hour  time  lapsing  could  not 
be  determined  in  this  evaluation. 

RESPONSES  TO  THE  GOES  SYSTEM  FINAL  INTERVIEW  QUESTIONNAIRE. 

1.  Do  you  feel  that  the  various  image  types  and  movements  of  those  images  are  an 
aid  to  efficient  briefing? 

Of  the  24  evaluators,  23  indicated  the  GOES  images  were  an  aid  to  more 
efficient  briefings.  Surprisingly,  the  only  specialist  who  replied  negatively 
was  both  GOES  data  trained  and  qualified  and  was  using  the  products  in  her  home 
facility.  This  specialist  stated  that  without  adequate  knowledge  of  image  inter¬ 
pretation,  it  would  be  difficult  to  use  GOES  products  for  more  than  general 
briefing  or  to  basically  self-brief. 

2.  Do  you  think  that  any  images  are  excessive  or  redundant  (such  as  combining 
visual  and  infrared)? 

No.  The  group  felt  the  two  image  types  complimented  each  other  and,  of  the 
two,  the  infrared  was  the  better  product. 

3.  Would  this  system  or  a  similar  system  add  to  your  ability  to  self-brief? 
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There  was  unanimous  agreement  that  it  would  add  to  one's  ability  to  self-brief 
(add  to  but  not  substitute  for  a  complete  preduty  briefing).  The  animated  system 
"sticks  in  your  mind  better"  and  as  one  specialist  succinctly  stated,  "The  differ¬ 
ence  is  sort  of  like  trying  to  explain  a  cartoon  compared  to  watching  an  animated 
version  of  the  same  cartoon." 

4.  What  comments,  if  any,  would  you  make  regarding  the  amount  of  information  that 
is  displayed  on  the  screen  at  one  time? 

The  information  was  adequate  and  very  appropriate  to  specialists'  needs  within 
an  FSS.  Suggestions  were  made  to  add  color  enhancements  and  overlay  maps.  One 
excellent  observation  made  regarding  the  information  displayed  was  the  ability, 
using  GOES  imagery,  of  interpreting  weather  between  reporting  points  (especially 
thunderstorm  activity). 

5.  If  you  had  a  zoom  capability,  what  would  be  your  preference;  e.g.,  a  400-  to 
600-mile  radius?  Also,  would  you  like  this  displayed;  i.e.,  ability  to  select  the 
area  of  the  country  you  want? 

Responses  tended  to  be  rather  broad  with  the  range  of  zoom  capability  of  200 
to  1,000  miles.  Specialists  indicated  a  large  range  due  to  their  specific  require¬ 
ments  during  a  particular  route  briefing.  The  ability  to  select  the  area  of  the 
country  they  desired,  and  the  adjustable  zoom  feature  were  features  highly  desired 
by  all  specialists. 

6.  Is  there  anything  you  want  to  add  or  comment  on? 

This  question  became  an  open  forum  during  the  interview,  and  specialists  were 
able  to  respond  broadly  to  the  prototype  system  and  the  machine  display  of  satel¬ 
lite  imagery  information.  Several  comments  are  worthy  of  note  and  emphasis: 

"Satellite  imagery  is  a  tool  to  be  used  with  other  weather  informational 
products  for  overall  effectiveness.  I  would  like  to  see  a  way  to  add  more 
information  to  GOES  products.  I  would  like  to  see  an  overlay  capability;  the 
animation  capability  is  tremendous.  I  prefer  the  step  feature  in  animation  because 
it  allows  you  to  work  at  your  own  rate.  I  definitely  prefer  a  pace  you  can 
control;  namely,  slow  speed  and  step  forward  and  back.  I  am  most  impressed  with 
the  animation  feature,  especially  for  trending.  I  also  like  the  "both"  function. 
The  "still"  function  is  also  good,  it  gives  you  the  most  current  visual  or  infrared 
picture.  The  capabilities  of  the  system  are  especially  good  in  the  summertime 
during  thunderstorm  activity  and  squall-line  development.  This  is  where  the 
trending  function  is  good;  with  hardcopy,  the  map  must  be  passed  around  as  needed 
(only  one  copy),  and  if  it  is  misplaced,  damaged,  etc.,  everyone  is  out  of  infor¬ 
mation;  but  with  the  CRT  display,  GOES  imagery  is  available  to  all  positions,  thus, 
a  most  important  consideration  for  me  in  my  facility." 

In  summary,  the  prototype  display  system  was  well  received  by  all  participants  in 
this  evaluation.  They  enjoyed  and  were  excited  when  talking  about  the  system.  A 
persistent  question  they  continually  asked  was  "It's  great  —  how  long  must  we  wait 
before  we  can  have  it?" 
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RESULTS 


The  synopsis  of  questionnaire  data  revealed  the  following  specialist  preferences, 
relating  to  satellite  pictorial  products: 

1.  Machine  (CRT)  versus  hardcopy  display  of  information. 

2.  Complete  adequacy  of  the  machine  display  in  the  performance  of  preflight 
briefing  dut  ies . 

3.  CRT  highly  preferred  over  television  display. 

4.  No  advantage  of  color  over  black  and  white  displays  (generally  a  neutral 
feeling  when  discussing  color  enhancements). 

3.  Digitally-enhanced  infrared  images  preferred  together  with  the  capability  to 
project  these  images  at  medium  speed,  slow  speed,  and  individually  (step  feature); 
the  speed  depending  on  one's  current  requirements. 

6.  Need  to  have  GOES  products  available  at  all  briefing  positions.  With 
hardcopy,  you  just  have  it  available  at  one  position,  and  it  must  be  handed  around, 
thereby  being  subject  to  damage  (ripped,  coffee  stained  just  where  you  need  to 
see  the  detail,  etc.). 

7.  Elimination  of  BOTH  modes  in  the  FAST  forward  and  reverse  speeds.  This  rapid 
"flip-flop"  effect  provided  no  meaningful  transfer  of  information  and  proved  to  be 
a  distraction  to  the  specialist. 

Animation  was  found  to  be  the  feature  most  praised  and  desired  in  utilising  GOES 
satellite  products.  As  one  GOES  trained  specialist  observed,  "The  animated  system 
is  much  better  than  an  oral  briefing.  It  sticks  in  your  mind  better." 

Another  specialist,  one  from  an  automated  facility  (but  not  GOES  system  trained), 
also  stated  it  succinctly,  "I'm  most  impressed  with  the  animation  feature, 
especially  for  trending.  The  capabilities  of  the  system  are  especially  good  in  the 
summertime  during  thunderstorm  activity  and  squall-line  development.  This  is  where 
the  trending  function  is  good." 

Based  on  the  personal  interviews  conducted,  the  step  feature  was  the  second  most 
highly  desired  function.  This  feature  was  preferred  in  animation,  for  both 
infrared  and  visual  because  it  allows  the  specialist  to  work  at  his  or  her  own 
pace.  It  is  liked  because  it  can  be  (individually)  controlled  (solely  by  the 
specialist).  The  rate  can  be  controlled  along  with  the  direction,  both  forward  and 
backward. 

Special  features  highly  praised  or  desired  are  as  follows: 

1.  Animation 

2.  Update  capability 

3.  Step  sequencing 
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4.  Area  sectorization/zoom  capability 

5.  Overlay  capability;  e.g.,  radar  summary,  fronts,  highs  and  lows 

6.  Ability  of  GOES  to  "f il 1-in-the-gaps"  between  weather  reporting  points 

7.  Grey  resolution  preferred  over  green  (background)  resolution 

8.  Selectable  zoom  (100  to  600  miles)  and  area  of  country  desired 

9.  HALT  feature 

Considering  the  responses  to  the  final  interview  questionnaire  and  the  personal 
interviews,  the  current  composite  evaluation  of  GOES  imagery  reveals  a  presently 
designed  system  fully  capable  of  meeting  the  specialist's  needs  to  perform  an 
effective  and  efficient  briefing.  The  specialist  now  has  an  additional  tool  to  aid 
and  enhance  pilot  briefings;  one  that  is  easy  to  learn  to  use,  extremely  fast  in 
the  transfer  of  information  to  the  user,  and  timely  in  the  rapid  gathering  and 
deployment  of  information  (30  minutes  from  orbital  source  to  specialist  and  pilot). 

When  added  to  the  specialist's  present  methods  and  tools  utilized  in  providing 
pilot  weather  briefings,  GOES  products  were  rated  a  significant  improvement  on  all 
aspects  rated  when  compared  to  no  change  from  the  present  system  (except  for 
item  14,  amount  of  eyestrain).  (See  figure  1.) 

Animation  was  the  single  feature  most  desired  and  most  highly  praised  in  this  GOES 
evaluation.  In  utilizing  this  tool,  the  specialist  desired  to  control  the  speed  of 
this  animation  (medium,  slow,  and  step)  to  suit  the  current  requests  and  require¬ 
ments  of  the  pilot  briefing.  Training  was  also  found  to  be  of  critical  importance 
in  the  full  utilization  of  the  GOES  products. 


CONCLUSIONS 


Taking  the  position  that  a  similar  system  in  a  real-time  environment  would  perform 
at  a  level  comparable  to  that  of  the  laboratory  evaluation,  it  is  reasonable  to 
conclude  that: 

1.  The  proposed  Geostationary  Operational  Environmental  Satellite  (GOES)  system 
was  found  to  be  an  effective  tool  when  used  with  other  information  and  products  for 
total  briefing  effectiveness  as  the  statistically  significant  results  of  the  T 
score  analysis  (table  2)  have  shown. 

2.  The  animation  capability  is  the  most  outstanding  feature  in  the  present  system 
evaluation. 

3.  The  step  mode  is  the  preferred  speed  medium  when  operating  the  system  for  use 
in  pilot  briefings. 

4.  Eyestrain  was  the  only  negative  aspect  identified  in  the  evaluation  of  the 
proposed  GOES  system. 
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RECOMMENDATIONS 


1.  It  is  recommended  that  additional  field  evaluations  be  conducted  to: 

a.  Sample  a  larger  population  of  specialists  having  the  required  knowledge 
and  skills, 

b.  Determine  operational  suitability  and  reliability  of  design, 

c.  Assess  the  overall  attitude  of  the  sample  population  towards  this  system, 

and 

d.  Determine  the  optimum  enhancements  required  by  the  user. 

2.  Any  limitations  identified  during  the  field  test  would  allow  management  and 
project  personnel  to  design  an  adaptation  strategy  to  overcome  these  limitations 
prior  to  the  specificat ion  and  procurement  phase  of  product  development.  Specifi¬ 
cally,  additional  tests  should  be  conducted  to  reduce  the  amount  of  eyestrain 
encountered  in  the  system  as  presently  designed. 

3.  Of  all  the  aspect  items  addressed  in  this  study,  as  well  as  other  Flight 
Service  Station  (FSS)  automation  studies,  "amount  of  eyestrain"  continues  to 
appear  as  a  negative  response,  thus  requiring  a  strong  recommendation  for  addi¬ 
tional  test  of  man-machine  relationship  to  reduce  the  impact  of  "eyestrain"  with 
relation  to  automation. 
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APPENDIX  A. 


GOES  SYSTEM  EVALUATION  QUESTIONNAIRE 


1.  Do  you  prefer  machine  display  (as  in  the  IPS  and  ARVIN  systems)  or  hardcopy  in 
the  display  of  satellite  weather  information  for  use  in  pilot  briefings? 

2.  Can  you  adequately  use  the  machine  display  medium  in  the  performance  of  your 
duties  at  preflight. . .at  in-flight. . .at  EFAS? 

3.  What  is  your  reaction  to  the  system's  capability  to  update  satellite  images? 

4.  What  is  your  reaction  to  the  sequencing  capability  of  the  system;  i.e.,  Do  the 

animated  displays  assist  you  in  weather  trending? 

5.  What  would  be  your  requirements  for  optimum  system  trending? 

6.  How  does  the  quality  of  the  pictures  displayed  on  the  CRT  and  TV  monitor 
compare  with  the  quality  of  the  pictures  off  the  Laserfax? 

7.  What  is  your  preference  as  a  display  medium,  CRT,  or  TV  monitor,  and  why? 

8.  Does  color  enhance  or  detract  from  the  usability  of  the  weather  product  when 

compared  to  standard  black  and  white  displays? 

9.  What  other  weather  pictorial  overlays  would  best  supplement  the  information  on 
the  satellite  images? 

10.  What  is  your  reaction  to  the  zooming  capability  of  the  system,  and  for  what 
portion  of  the  product  to  be  displayed? 

11.  Please  indicate  by  circling  your  response. 


a.  The  rate  of  sequencing  you  prefer 

1.  fast  (10-20  frames  per  second) 

2.  medium  (4-9  frames  per  second) 

3.  slow  (2-3  frames  per  second) 

b.  The  degree  of  sequencing  you  prefer 

1.  3-hour  time  lapsing 

2.  6-hour  time  lapsing 

3.  12-hour  time  lapsing 

4.  24-hour  time  lapsing 


c.  Which  do  you  find  easier  to  work  with 

1.  'tgitally  enhanced  infrared  isMges 

2.  standard  video  products 
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APPENDIX  B 


GOES  SYSTEM  FINAL  INTERVIEW  QUESTIONNAIRE 


Special ist  t raining/ qua 1 i fi cat  ions : 

EFAS  training:  Yes _  No _ 

Are  you  currently  certified? _ 

Approx,  time  used _ _ _ 

GOES  training:  Yes _ No _ 

Are  you  presently  using  these  products? 
Approx,  time  used _ _ _ 

Automation  Experience:  FSS  Facility _ 

Manual  (non automated) _ 

Service  A _ _  _ 

MAPS 

AWANS 


1  •  Do  you  feel  that  the  various  image  types  and  movements  of  those  images  are  an 
aid  to  efficient  briefings? 

2.  Do  you  think  that  any  image  types  are  excessive  or  redundant,  such  as  combining 
visual  and  infrared? 

3.  Would  this  system,  or  a  similar  system,  add  to  your  ability  to  self-brief? 

4.  What  comments,  if  any,  would  you  make  regarding  the  amount  of  information  that 
is  displayed  on  the  screen  at  one  time? 

5.  If  you  had  a  zoom  capability,  what  would  be  your  preferences;  i.e.,  a  400-600 
mile  radius,  or  what?  Also,  would  you  like  this  displaced;  i.e.,  ability  to  select 
the  area  of  the  country  you  want? 

6.  Is  there  anything  you  want  to  add  or  comment  on? 
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no  a  prion  reason  to  expect  deviations  from  the  mean  m  only  a  single  direction, 
two-tailed  tests  were  used  for  all  items.  All  questions  were  phrased  so  that 
careful  reading  would  result  in  fair  responses,  thus  precluding  stereotype  answers. 
Also,  response  categories  were  varied  so  as  to  minimize  common  sources  of  error, 
such  that  no  response  was  unduly  influenced  by  a  single  or  overall  favorable  aspect 
when  rating  dimensions.  That  is,  the  form  was  designed  so  the  respondents  would 
read  and  respond  to  each  item  independent  of  the  previous  item  or  any  overall 
attitude  toward  an  automated  system. 

The  strong  preference  for  such  a  display  system  is  profoundly  evident  from  the 
statistical  results  presented  in  tables  2  and  3.  Although  the  questions  were 
phrased  to  include  a  balance  of  favorable  or  unfavorable  responses,  the  interpreted 
value  of  the  changes  was  positive  over  93  percent  of  the  survey.  The  aspect  item 
that  yielded  the  highest  t  score  was  item  1  (speed  of  data  access). 

SUMMARY  OF  RESPONSES. 

RESPONSES  FROM  GOES  SYSTEM  EVALUATION  QUESTIONNAIRES. 

1.  Do  you  prefer  machine  display  (as  in  the  IPS)  or  hardcopy  display  of  satellite 
weather  information  for  use  in  pilot  briefings? 

Machine  display,  utilizing  CRT's,  was  the  overwhelming  choice  as  a  display 
medium.  Hardcopy  had  a  singular  benefit  mentioned  by  several  specialists;  namely, 
increased  clarity  especially  on  the  visual  images.  But,  even  with  this  benefit, 
those  specialists  prefer  machine  display  when  looking  at  the  total  system 
utilization.  Pertinent  comments  were  as  follows: 

"The  machine  display  is  much  better  suited  for  pilot  briefings  because  of  the 
looping  capabilities,  ease  in  accessibility,  and  simple  operational  codings." 

"1  feel  the  IPS  (machine  display  under  evaluation)  is  better  suited  for 
briefings  as  it  is  much  easier  to  show  a  trend  and  has  an  easier  access  to  weather 
data . " 

"Machine  display  does  not  have  the  clarity  of  hardcopy,  particularly  with 
visual  (VI)  imagery.  However,  advantages  of  machine  display;  i.e.,  "movie  loop," 
outweigh  this  factor." 

"1  prefer  a  machine  system.  In  normal  situation,  the  hardcopies  float  around 
various  positions  making  access  difficult,  if  not  impossible." 

2.  Can  you  adequately  use  the  machine  display  medium  in  the  performance  of  your 
duties  at  pref 1 ight — at  in-f 1 ight . . . at  EFAS? 

Response  to  this  question  was  mixed:  73  percent  said  "Yes;"  9  percent  said 
"No;"  and  18  percent  gave  a  conditional,  but  qualified  "Yes."  The  "no"  responses 
were  due  to  lack  of  training/familiarization  associated  with  interpreting  satellite 
imagery . 

Responses  were  highly  favorable  at  the  preflight  position  (100  percent), 
especially  in  face-to-face  briefings.  Responses  were  somewhat  negative  at 
in-flight  and  EFAS.  As  one  GOES  system  trained  specialist  said,  "Real-time  demands 


were  not  met."  The  greatest  value  seemed  to  be  in  the  specialist  self-briefing  for 
an  overall  view  of  the  weather  trend.  In  addition,  it  was  felt  that  GOES  product 
could  be  effectively  used  at  all  positions  and  would  augment  weather  graphical 
products;  e.g.,  surface  analysis  charts. 

3.  What  is  your  reaction  to  the  system's  capability  to  update  satellite  images? 

Responses  ranged  from  "very  good"  to  "excellent."  Update  prior  to  machine 
display  was  left  to  comparison  only.  In  utilizing  the  CRT,  the  movement  leaves 
little  doubt. 

4.  What  is  your  reaction  to  the  sequencing  capability  of  the  system?  Do  the 

animated  displays  assist  you  in  weather  trending? 

The  animation  feature  was  the  most  highly  praised  feature  identified  in  this 
GOES  evaluation.  The  animation  and  the  capability  to  control  one's  own  pace 
utilizing  the  step  mode  were  the  major  advantages  identified  in  this  experimental 
GOES  system.  The  animated  displays  were  noted  as  being  a  definite  advantage  (1)  in 
seeing  a  weather  trend,  (2)  for  developing  an  insight  into  weather  causes,  and 
(3)  being  able  to  observe  real-time  weather  moving  over  an  area  (e.g.,  thunderstorm 
development) . 

5.  What  would  be  your  requirement  for  optimum  system  trending? 

Comments  to  this  question  were  broad  and  varied.  Since  21  of  the  24 
specialists  had  not  had  GOES  training,  their  comprehension  of  the  question  was 
limited.  Hence,  only  the  responses  from  those  with  the  formal  training  in 
satellite  interpretation  were  considered.  Their  suggestions  were  for  color 
enhancements,  sectorizing,  and  sequencing  at  the  local  level  and  the  availability 
of  hardcopy,  as  well  as  machine  display. 

6.  How  does  the  quality  of  the  pictures  displayed  in  the  CRT  and  TV  monitor 
compare  with  the  quality  of  the  pictures  from  the  Laserfax? 

It  was  stated  that  when  looking  at  a  single  image  (picture),  the  quality  was 
better  than  on  the  Laserfax.  It  was  more  difficult  to  determine  fine  shade  differ¬ 
ences  on  the  CRT.  It  was  suggested  that  a  change  in  the  CRT  phosphor  from  green  to 
grey  (black/white)  would  improve  the  present  display.  The  CRT  picture  seemed  to 
quiver  slightly,  causing  some  eyestrain  when  viewed  closely.  It  should  be  noted 
that  these  comments  relate  to  individual  or  single  picture  quality,  not  the  overall 
evaluation  of  the  GOES  system. 

7.  What  is  your  preference  as  a  display  medium  —  the  CRT  or  the  TV  monitor  — 
and  why? 

The  CRT  was  preferred  over  the  TV  monitor  due  to  picture  quality  and 
resolution. 

8.  Does  color  enhance  or  detract  from  the  usability  of  the  weather  product  when 
compared  to  standard  black  and  white  displays? 


10 


